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Two of a Kind: Are Your Districts’ Evaluation 
Systems Equivalent? 


Question From the Field 


How are states creating equivalency processes for district evaluation systems? 

States in the midst of implementing evaluation reforms face a common dilemma: how to ensure 
that all your districts are implementing quality educator evaluation systems while still providing 
them with the flexibility to design systems best suited to their own unique needs. One answer is 
to create an equivalency process (also known as an approval process) to verify that districts meet 
state requirements for evaluation components and processes. An equivalency process allows 
a state to provide quality assurance for district evaluation plans. States review documentation 
and evidence submitted by a district about its evaluation system. In designing an equivalency 
process, states typically include clear guidance on the requirements of the evaluation system, 
recommendations on best practices, rubrics or processes for meeting the requirements, and 
examples of approved models or components. 

The nuts and bolts for creating an equivalency process, however, can seem daunting to sort out. In 
response to questions from the field, we developed a set of promising practices and recommendations 
for consideration when developing an equivalency process in your own state. 


r Q ESTABLISH GUIDELINES AND SPECIFY ALLOWABLE FLEXIBILITY 
Measuring Student Learning 


■ Provide clear, comprehensive guidelines for how 
student learning should be measured for both 
tested and nontested grades and subjects. 

■ Create a uniform process for calculating 
student learning for all districts that accounts 
for several assessments for different grades 
and subjects. 

■ Carefully consider the availability of data and district capacity when deciding how much 
flexibility to give districts in calculating student learning. 


PAGE 1 


U If you opt to address tested 

subjects and grades by using 
^ state standardized assessments 

and a uniform growth calculation, be 
prepared to engage stakeholders in the 
process. Educators need support in 
understanding, interpreting, and using 
a growth score generated by the state. 



State 

^Spotlight 


Rhode Island provides clear guidelines on how student growth is calculated for tested grades, using the state’s 
student growth percentile model, as well as how to use student learning objectives (SLOs) to measure student growth for 
all educators, including teachers of nontested grades and subjects. The SLO process in Rhode Island allows districts 
to choose assessments that best fit each educator’s position while also maintaining rigor and comparability between 
growth scores. 


Source: Student Learning Objectives (http://www.ride.ri.gov/educatorquality/educatorevaluation/SLO.aspx) 



MAKE IT CRYSTAL CLEAR 
Practice Measures and Rubrics 


Issue clear guidance to districts on the requirements 
for assessing educator practice. This approach will help 
ensure that the measures of practice are both rigorous 
and of high quality. In developing this guidance, consider 
the following suggestions: 

■ Require that practice rubrics align with state 
teaching and leadership standards as well as 
standards for college and career readiness. 

■ Require that educator practice be scored on a 
similar scale statewide (e.g., observation rubrics 
are based on a four-level performance scale). 

■ State upfront whether districts can include 
district-selected measures of practice or 
must use or select from a state-determined 
set of measures. 

■ Require districts to include their plans for 
training observers and assessing rater 
reliability and accuracy in using the 
instructional or leadership rubrics to 
assess educator practice. 


I If districts can select 

additional measures of 
^ practice, be sure to provide 

clear guidelines on what types of measures 
are appropriate, including their scope. 

If districts can select their own 
observation rubrics, create a list of 
state-approved vendors or instruments 
and include rubrics already in use in 
districts in your state. Be sure to specify 
that districts should either (1) select a 
research-based rubric that has been 
validated in a variety of educational 
contexts or (2) include plans to validate 
the rubric in their own district. 


Ohio has developed a Web-based alignment tool that districts may use to perform a gap analysis between their 
district-designed system and the state’s requirements. Districts may use this tool to submit their final evaluation 
system for approval when completed. If districts use a district-created performance rubric or have modified an 
existing state-approved rubric, the tool allows districts to create a crosswalk between the state’s required standards 
and indicators and the district’s rubric. The tool generates a report for districts that describes the areas in which the 
district rubric does not meet state requirements. This feature ensures that districts can demonstrate equivalency 
before they submit their evaluation system for final state review. 


Sources: Gap Analysis Tool (http://education.ohio.gov/GD/Templates/Pages/ODE/ODEDetail. 
aspx?page=3&TopicRelationlD=1230&ContentlD=125741); Ohio ESEA Flexibility Request (http://www2.ed.gov/ 
policy/eseaflex/approved-requests/oh.pdf) 


PAGE 2 




3 


MAKE IT UNIFORM 
Rating Requirements 


Summative evaluation ratings need to be comparable across a state. The majority of states require 
that districts use a uniform scale with established performance levels for all summative scores 
(e.g., basic, developing, proficient, and distinguished). States that allow flexibility in the scale or 
performance labels typically require a district to translate its scale into the statewide minimum 
rating scale. 


Arizona allows districts to determine the number and the labels of performance ratings for summative scoring; however, 
districts that use more than four performance levels must be able to condense their ratings into a four-level scale for 
state reporting purposes. 

Source: Arizona Framework for Measuring Educator Effectiveness (http://www.azed.gov/wp-content/uploads/PDF/ 
ArizonaFrameworkforMeasuringEducatorEffectiveness.pdf) 


.- El MAKE TIME 

Feedback, Revisions, and Tools in the Process 


When designing the structure, the sequence, and 
the timing of an equivalency process, be sure to 
consider both state education agency (SEA) and 
district capacity. Districts can benefit from a 
quick, one-month approval process that allows 
them to focus on implementation and planning. 

States might consider a presubmission process 
that allows districts to receive early SEA support 
before formal submission; SEA capacity, however, 
must be carefully considered in deciding whether 
to include these features. Presubmission 
processes, shorter timelines, and assessing multiple documents may tax SEA staff or may not be 
feasible in states with large numbers of districts. Regardless of capacity considerations, be sure 
to build sufficient time into the equivalency process for districts to make revisions and engage 
with stakeholders. 

States have developed several strategies for addressing timing and capacity considerations: 

■ Add a presubmission process whereby districts can receive informal feedback from the 
state on their draft applications. 

■ Ask districts to submit an intent-to-apply letter in advance of the presubmission deadline. 
This notice allows a state to plan sufficient time and resources for reviewing district 
applications in a timely manner. 


Build requirements for validity testing 
into the equivalency process. Issue 
^ guidance on how districts should 

develop plans for system testing and evaluation; 
be sure to include information on how the state 
intends to gather data on the quality of systems 
across the state and over time. Quality assurance is 
a critical guiding principle throughout this process. 
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■ Provide opportunities for districts to modify and revise their applications after their initial 
submission. 

■ Create a preassessment audit for districts to use as they design their evaluation systems, 
which could include checklists, rubrics, and self-appraisal tools. 

■ Allow districts to submit preassessment tools with document citation rather than requiring 
a narrative. This practice helps support districts with limited capacity by saving time spent 
on design and documentation. 



Rhode Island 

■ Rubric assessment. The state uses educator evaluation standards to assess district-created evaluation systems 
and has developed a rubric for this process. The rubric rates districts on the extent to which the district’s system 
has met the standards for each educator evaluation standard and substandard. 

■ District self-assessment and guidelines. These tools assist districts that choose to design their own evaluation 
systems. The guidelines document describes the process intuitively for districts to clarify the process. 

■ Reducing the burden. Rhode Island has attempted to reduce the documentation burden on districts by allowing 
them to simply cite the document name and page number of manuals or training documents rather than rewriting 
or summarizing the document. The state also provides forms for districts to complete to describe the ways in which 
all educators are evaluated. 


Source: District Developed Evaluation Systems (http://www.ride.ri.gov/educatorquality/educatorevaluation/DDS.aspx) 


Tennessee allows districts to choose to either modify the state model to fit their needs or create a district-designed 

W I yXfUl j3f 

r, j|M|k alternate evaluation system. Districts selecting either of these options must submit an application to the Tennessee 
Department of Education for review in the spring. The department shares feedback and works with the district over the 
month following the submission to determine whether approval is warranted. Districts must submit their model by 
March 15, and the department will provide notice of decision by April 15. 

Sources: Evaluation Model Selection Grand Division Meetings (http://team-tn.org/assets/misc/Evaluation%20 
Model%20Selection%20Presentation.pdf); Model Selection Overview (http://team-tn.org/assets/misc/Model_ 
Selection_Overview.pdf) 



Massachusetts requires districts that opt to design their own systems to complete an Educator Evaluation Review 
Questionnaire, which is not yet publicly available in its full form. The questionnaire includes yes/no questions and 
clarifies which aspects are requirements of the evaluation system and which aspects are optional. Districts must submit 
evidence from bargaining contracts and other district-created documents to support their explanation and justifications 
for the evaluation system design. All districts must submit the form, although the length and the detail required vary 
according to how many modifications the district has made to the model evaluation system. 


Source: Massachusetts Model System for Educator Evaluation, Part I: District-Level Planning and Implementation Guide 
(http://www.doe.mass.edu/edeval/model/Partl.pdf) 
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Bonus 

Resource 


STATE EQUIVALENCY PROCESS CHECKLIST 


In developing a district equivalency process, use the following list as a starting point to create your own checklist or rubric 
that both SEA and district-level staff can use to verify alignment between state requirements and district designs. 


Stakeholder Engagement and Communication 

Has the district completed the following tasks related to stakeholder engagement and communication? 

□ Established and used a stakeholder committee to design and implement the evaluation system. 

□ Developed a communication plan that ensures that all educators are aware of and informed about 
the new evaluation system. 

□ Provided multiple opportunities for educators to offer input and feedback on the evaluation system design 
and implementation plan. 


Performance Rubrics 

Has the district completed the following tasks related to performance rubrics? 

□ Selected or created a rubric to evaluate teacher and leader practice. 

• Are the rubrics on the state’s approval list? 

• If a district is using a rubric not on the approved list, has it demonstrated that the rubric aligns with 
state standards of teaching practice or leadership? 

• Does the rubric align with the state’s system in terms of the number and the descriptions of levels of 
performance (e.g., basic, developing, proficient, and distinguished)? 

□ Developed a plan for training evaluators to use the rubrics with a high degree of accuracy and reliability. 

Measuring Student Growth 

Has the district completed the following tasks related to measuring student growth? 

□ Completed a thorough review of its data systems to ensure that data exist to accurately link teachers 
to students. 

□ Developed a plan to use state-provided results from standardized tests in teacher evaluation scores. 

• Developed a process to ensure accuracy, such as teacher verification of class rosters. 

• Used state guidance and requirements to develop processes for measuring teachers’ contributions 
to student learning growth in nontested subjects and grades. 

• Determined procedures for providing guidance and monitoring to teachers in selecting, scoring, and 
reporting results from local assessments. 

□ Determined how the student growth component will fit into the overall evaluation system and performance rating. 

□ Developed training and communication plans to assist staff in understanding growth scores, learning how to use 
student learning data, and identifying or designing assessments. 
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Summative Ratings 

Has the district completed the following tasks related to summative ratings? 

□ Identified a process for determining summative evaluation ratings that meet state requirements (e.g., scoring 
procedures or weights of measures). 

□ Selected summative evaluation rating labels that align with state requirements. 

□ Developed a plan for collecting, securing, and analyzing educators’ summative evaluation ratings. 

Piloting, Validating, and Evaluating the System 

Has the district completed the following tasks related to piloting, validating, and evaluating the system? 

□ Developed a piloting process prior to rolling out the system for full implementation with high-stakes 
consequences. 

□ Developed a plan for validating the system by examining trends in correlations between measures of educator 
practice (e.g., observation, portfolios, and perception surveys) and measures of student learning. 

□ Developed a plan for periodically evaluating the system by gathering feedback from stakeholders and educators 
on the validity and the utility of the evaluation system for both evaluating educator effectiveness and supporting 
educators’ professional growth. 


I WANT TO KNOW MORE ! 

For more examples or information on this topic, please e-mail gtlcenter@air.org. 

Catherine Jacques is a research associate at American Institutes for Research and provides technical assistance 
for the Center on Great Teachers and Leaders. 


1351_03/13 


PAGE 6 



